Winograd Schemas and Machine Translation

A Winograd schema is a pair of sentences that differ in a single word and that contain an ambiguous pronoun whose referent is different in the two sentences and requires the use of commonsense knowledge or world knowledge to disambiguate. This paper discusses how Winograd schemas and other sentence pairs could be used as challenges for machine translation using distinctions between pronouns, such as gender, that appear in the target language but not in the source. A Winograd schema (Levesque, Davis, and Morgenstern 2012) is a pair of sentences, or of short texts, called the elements of the schema, that satisfy the following constraints: 1. The two elements are identical, except for a single word or two or three consecutive words. 2. Each element contains a pronoun. There are at least two noun phrases in the element that, grammatically, could be the antecedents of this pronoun. However, a human reader will reliably choose one of these as plausible and reject the rest as implausible. Thus, for a human reader, the resolution of the pronoun in each element is unambiguous. 3. The correct resolution of the pronoun is different in the two sentences. 4. Computationally simple strategies, such as those based on single word associations in text corpora or selectional restrictions, will not suffice to disambiguate either element. Rather, both disambiguation require some amount of world knowledge and of commonsense reasoning. The following is an example of a Winograd schema: A. The trophy doesn't fit in the brown suitcase because it's too large. B. The trophy doesn't fit in the brown suitcase because it's too small.